http/1.1 decompress

[Frederik Vanderhaegen]

Hi,

I'm writing an application who 'sniffs' port 80.
The sniffing works 'perfect', I receive a byte array with the data sent over
the port.
My problem is the following, when I write the array to a txt file most of
the content is encoded.
The only things that are readable are the http headers from which I can read
that Internet Explorer uses "content-encoding: gzip, deflate".

So I may conclude that I'll need to decompress the data I received.
This is easier said than done:).
I tried to decompress with GZipStream, DeflateStream (both from .NET 2.0)
and SharpZipLib.Gzip.GzipInputstream but neither of them want to decompress
my array.
In the beginning I received several errors about an incorrect magicnumber
and incorrect format. I thought why not add the gzip header
(31,139,8,0,0,0,0,0,0,0) at the start of my array.
..Net didn't throw any exceptions but it didn't decompress (the output is now
an empty file because the bytesread of my decompress stream is 0) my data
either.

Does anyone knows how that Internet Explorer performs the decompression or
how I can perform the decompression myself?

Thx in advance

Frederik.

[Kevin Spencer]

You've run into a rather rare animal there, Frederik, although I suppose
that at some point it's likely to catch on popularly, to save bandwidth.

I found a good article on the subject, which also links to some libraries
you can use:

http://www.15seconds.com/issue/020314.htm

--
HTH,

Kevin Spencer
Microsoft MVP
Chicken Salad Surgery

It takes a tough man to make a tender chicken salad.
"Frederik Vanderhaegen" <fr*******************@pandora.bewrote in message
news:O5**************@TK2MSFTNGP05.phx.gbl...

Hi,

I'm writing an application who 'sniffs' port 80.
The sniffing works 'perfect', I receive a byte array with the data sent
over the port.
My problem is the following, when I write the array to a txt file most of
the content is encoded.
The only things that are readable are the http headers from which I can
read that Internet Explorer uses "content-encoding: gzip, deflate".

So I may conclude that I'll need to decompress the data I received.
This is easier said than done:).
I tried to decompress with GZipStream, DeflateStream (both from .NET 2.0)
and SharpZipLib.Gzip.GzipInputstream but neither of them want to
decompress my array.
In the beginning I received several errors about an incorrect magicnumber
and incorrect format. I thought why not add the gzip header
(31,139,8,0,0,0,0,0,0,0) at the start of my array.
.Net didn't throw any exceptions but it didn't decompress (the output is
now an empty file because the bytesread of my decompress stream is 0) my
data either.

Does anyone knows how that Internet Explorer performs the decompression or
how I can perform the decompression myself?

Thx in advance

Frederik.

[Frederik Vanderhaegen]

Kevin,

I don't seem to be able to solve the decompress problem but I have another
possible solution for my problem.
Maybe you can help me with that.
I should be able to remove the content-encoding field from the header before
the request is send to the server =the server won't compress the message
and I can read the response.
So I think I should create a sort of proxy that checks outgoing messages for
the content-encoding field.

I should capture the messages before they're are being send to port 80,
adjust them and do a resend.

The only problem is that I'm not sure how I can create such a construction.

Do you have an idea how I can start?
Thx in advance

Frederik
"Kevin Spencer" <uc*@ftc.govschreef in bericht
news:Of**************@TK2MSFTNGP04.phx.gbl...

You've run into a rather rare animal there, Frederik, although I suppose
that at some point it's likely to catch on popularly, to save bandwidth.

I found a good article on the subject, which also links to some libraries
you can use:

http://www.15seconds.com/issue/020314.htm

--
HTH,

Kevin Spencer
Microsoft MVP
Chicken Salad Surgery

It takes a tough man to make a tender chicken salad.
"Frederik Vanderhaegen" <fr*******************@pandora.bewrote in
message news:O5**************@TK2MSFTNGP05.phx.gbl...

>Hi,

I'm writing an application who 'sniffs' port 80.
The sniffing works 'perfect', I receive a byte array with the data sent
over the port.
My problem is the following, when I write the array to a txt file most of
the content is encoded.
The only things that are readable are the http headers from which I can
read that Internet Explorer uses "content-encoding: gzip, deflate".

So I may conclude that I'll need to decompress the data I received.
This is easier said than done:).
I tried to decompress with GZipStream, DeflateStream (both from .NET 2.0)
and SharpZipLib.Gzip.GzipInputstream but neither of them want to
decompress my array.
In the beginning I received several errors about an incorrect magicnumber
and incorrect format. I thought why not add the gzip header
(31,139,8,0,0,0,0,0,0,0) at the start of my array.
.Net didn't throw any exceptions but it didn't decompress (the output is
now an empty file because the bytesread of my decompress stream is 0) my
data either.

Does anyone knows how that Internet Explorer performs the decompression
or how I can perform the decompression myself?

Thx in advance

Frederik.

[Joerg Jooss]

Thus wrote Frederik,

Kevin,

I don't seem to be able to solve the decompress problem but I have
another
possible solution for my problem.
Maybe you can help me with that.
I should be able to remove the content-encoding field from the header
before
the request is send to the server =the server won't compress the
message
and I can read the response.
So I think I should create a sort of proxy that checks outgoing
messages for
the content-encoding field.
I should capture the messages before they're are being send to port
80, adjust them and do a resend.

The only problem is that I'm not sure how I can create such a
construction.

Do you have an idea how I can start?

To avoid receiving compressed content, make sure *not* to send an Accept-Encoding
header with a value of "gzip" or "deflate". .NET 1.x has no built-in support
for compression, so just make sure you don't set the header in your code
explicitly. In .NET 2.0, set HttpWebRequest.AutomaticDecompression to DecompressionMethods.None.

Cheers,
--
Joerg Jooss
ne********@joergjooss.de

[Kevin Spencer]

Hi Frederik,

I think I may have found something for you. Check out the articles in the
series below:

http://www.eggheadcafe.com/articles/20011226.asp

There is a compression library referenced in the articles that sounds like
what you need.

--
HTH,

Kevin Spencer
Microsoft MVP
Chicken Salad Surgery

It takes a tough man to make a tender chicken salad.
"Frederik Vanderhaegen" <fr*******************@pandora.bewrote in message
news:%2****************@TK2MSFTNGP02.phx.gbl...

Kevin,

I don't seem to be able to solve the decompress problem but I have another
possible solution for my problem.
Maybe you can help me with that.
I should be able to remove the content-encoding field from the header
before the request is send to the server =the server won't compress the
message and I can read the response.
So I think I should create a sort of proxy that checks outgoing messages
for the content-encoding field.

I should capture the messages before they're are being send to port 80,
adjust them and do a resend.

The only problem is that I'm not sure how I can create such a
construction.

Do you have an idea how I can start?
Thx in advance

Frederik
"Kevin Spencer" <uc*@ftc.govschreef in bericht
news:Of**************@TK2MSFTNGP04.phx.gbl...

>You've run into a rather rare animal there, Frederik, although I suppose
that at some point it's likely to catch on popularly, to save bandwidth.

I found a good article on the subject, which also links to some libraries
you can use:

http://www.15seconds.com/issue/020314.htm

--
HTH,

Kevin Spencer
Microsoft MVP
Chicken Salad Surgery

It takes a tough man to make a tender chicken salad.
"Frederik Vanderhaegen" <fr*******************@pandora.bewrote in
message news:O5**************@TK2MSFTNGP05.phx.gbl...

>>Hi,

I'm writing an application who 'sniffs' port 80.
The sniffing works 'perfect', I receive a byte array with the data sent
over the port.
My problem is the following, when I write the array to a txt file most
of the content is encoded.
The only things that are readable are the http headers from which I can
read that Internet Explorer uses "content-encoding: gzip, deflate".

So I may conclude that I'll need to decompress the data I received.
This is easier said than done:).
I tried to decompress with GZipStream, DeflateStream (both from .NET
2.0) and SharpZipLib.Gzip.GzipInputstream but neither of them want to
decompress my array.
In the beginning I received several errors about an incorrect
magicnumber and incorrect format. I thought why not add the gzip header
(31,139,8,0,0,0,0,0,0,0) at the start of my array.
.Net didn't throw any exceptions but it didn't decompress (the output is
now an empty file because the bytesread of my decompress stream is 0) my
data either.

Does anyone knows how that Internet Explorer performs the decompression
or how I can perform the decompression myself?

Thx in advance

Frederik.